Comparison between cephalometric measurements using digital manual and web-based artificial intelligence cephalometric tracing software

ABSTRACT Objective: The aim of this study was to compare the measurements performed with digital manual (DM) cephalometric analysis and automatic cephalometric analysis obtained from an online artificial intelligence (AI) platform, according to different sagittal skeletal malocclusions. Methods: Cephalometric radiographs of 105 randomly selected individuals (mean age: 17.25 ± 1.87 years) were included in this study. Dolphin Imaging software was used for DM cephalometric analysis, and the WebCeph platform was used for AI-based cephalometric analysis. In total, 10 linear and 12 angular measurements were evaluated. The paired t-test, one-way ANOVA test, and intraclass correlation coefficient tests were used to evaluate the differences between the two methods. The level of statistical significance was set at p< 0.05. Results: Except for SNB, NPog, U1.SN, U1.NA, L1-APog, I/I, and LLE parameters, all other parameters presented significant differences between the two methods (p< 0.05). While there was no difference (p> 0.05) in both SNA and SNB measurements between the two methods in the Class I malocclusion group, there was a difference between both methods in the Class II malocclusion group. Meanwhile, only the SNA in the Class III malocclusion group was different (p< 0.05). The ANB angle differed significantly in all three malocclusion groups. For both methods, all parameters except CoA and CoGn were found to have good correlation. Conclusion: Although significant differences were detected in some measurements between the two cephalometric analysis methods, not all differences were clinically significant. The AI-based cephalometric analysis method needs to be developed for more specific malocclusions.


INTRODUCTION
Cephalometric analysis performed using certain anatomical points on lateral cephalometric radiographs is an important tool that orthodontists use to plan their treatment and to monitor the development of growing individuals. [1][2][3] In the past, drawings used for lateral cephalometric analysis were performed manually on transparent tracing paper by orthodontists. 4 With the development of technology, these processes began to be carried out digitally, and significant improvements were achieved in terms of speed, quality, and reliability. 5 The digitization of cephalometric images facilitates the treatment planning phase, by eliminating the human error inherent in the stage of traditional X-ray radiographs preparation in dark rooms, and introducing the possibility of digitally storing and sharing images in a practical way. 1,5,6 Converting from a manual cephalometric analysis technique to a digital cephalometric analysis technique provides many advantages, but still results in time wasted in front of a computer screen and requires professional supervision. 5,7,8 Today, diagnostic procedures based on computer-aided artificial intelligence (AI) are increasing, especially in dentistry applications that require radiographic evaluation. 7, 9,10 In several studies, it has been stated that AI-based applications are useful in determining the points used in cephalometric analysis, and can be used for measurements based on cephalometric analysis. 3,7,11 For these reasons, studies on the reliability and usability of Çoban G, Öztürk T, Hashimli N, Yağcı A -Comparison between cephalometric measurements using digital manual and web-based artificial intelligence cephalometric tracing software 5 AI still provide insufficient evidence, and it is believed that research into more specific areas should be increased.
The clinical use of information technology in Orthodontics has increased significantly in recent years. Thus, the aim of this study is to compare AI-based cephalometric analysis and digital-manual-based cephalometric analysis (DM) in Orthodontics, and to evaluate the reliability of AI in different sagittal skeletal malocclusion classes.

MATERIAL AND METHODS
This study, conducted according to the principles of the Helsinki Declaration, was approved by the Erciyes University Clinical Research Ethics Committee (approval no: 2020/498). Prior to the study, informed consent was obtained from all participants and parents/guardians included in the study. In order to determine the results that could produce a significant difference in the study, according to the power analysis using G*Power (v. 3.0.10, Franz Faul, Universität Kiel, Germany) software, it was determined that a 0.05 significance level, 0.85 effect size, and sample size of 35 individuals for each group in 95% power would be sufficient. 12 Inclusion criteria of the study comprised the following: (1) individuals with dental and skeletal Class I, II, or III malocclusion, (2) patients with ideal pretreatment diagnosis and records, (3) patients without any congenital anomaly or dentofacial syndrome, and (4) cephalometric measurements Çoban G, Öztürk T, Hashimli N, Yağcı A -Comparison between cephalometric measurements using digital manual and web-based artificial intelligence cephalometric tracing software 6 taken before treatment. It was planned to select individuals with radiographs of adequate clarity and quality. The exclusion criteria were determined as: (1) insufficient diagnosis or inadequate individual records, (2) presence of any congenital anomaly or dentofacial syndrome, and (3) cephalometric radiographs taken before treatment whose inadequate clarity and quality made them unusable. Dolphin Imaging cephalometric analysis software (v. 11.5, California, USA) was used for DM analysis.
A web-based, free online cephalometric analysis service called WebCeph (WEBCEPH™, Artificial Intelligence Orthodontic & Orthognathic Cloud Platform, South Korea, 2020) was used for AI-based automatic cephalometric analysis.
The analysis required for the classification of skeletal malocclusion was carried out with Dolphin Imaging software 5 (Fig 1A).
Subsequently, the same cephalometric radiographs were loaded into the cloud-based WebCeph software 13 (Fig 1B), and analyses were performed using AI. The cephalometric points used in the study are presented in Figure 2, and illustrations of the linear and angular measurements are presented in Figure 3. The subgrouping of the individuals included in the study according to the sagittal skeletal malocclusion classification was determined by skeletal Class I (0 < ANB < 4), Class II (ANB > 4), and Class III (ANB < 0) in the sagittal dimension. 5,14 The cephalometric radio-   (Table 1).
All radiographs to be used in the study were taken by the same technician, using the same cephalometry device (OP100; Çoban G, Öztürk T, Hashimli N, Yağcı A -Comparison between cephalometric measurements using digital manual and web-based artificial intelligence cephalometric tracing software    The intraclass correlation coefficient (ICC) was used to evaluate the reliability between the two methods. 11 The statistical significance value was defined as p < 0.05.

METHOD ERROR
In order to evaluate the reliability of each of the cephalometric analysis methods, the measurements of 10 samples, randomly selected among all samples, were repeated two weeks after the first measurement by the same researcher for the DM method. A separate account was created for the AI method, and the radiographs were reloaded. The reliability analysis of the repeatability between the first and second measurements was evaluated using the ICC. Accordingly, the confidence interval for the DM method was found between 0.768 (95% confidence interval, lower bound) and 0.997 (95% confidence interval, upper bound), while it was between 0.940 (95% confidence interval, lower bound) and 0.999 (95% confidence interval, upper bound) for WebCeph.
Çoban G, Öztürk T, Hashimli N, Yağcı A -Comparison between cephalometric measurements using digital manual and web-based artificial intelligence cephalometric tracing software 13

RESULTS
Demographic data indicating the age and gender information of the individuals included in this study are presented in Table 1.
The data of the measurements examined with the digitally drawn cephalometric radiograph method and the AI-based automatic measurement method are presented in Table 2 Çoban G, Öztürk T, Hashimli N, Yağcı A -Comparison between cephalometric measurements using digital manual and web-based artificial intelligence cephalometric tracing software 14   Çoban G, Öztürk T, Hashimli N, Yağcı A -Comparison between cephalometric measurements using digital manual and web-based artificial intelligence cephalometric tracing software 15   found that there were differences in all three groups (Table 4).
An evaluation of the reliability between the two cephalometric analyzes is presented in Table 5. Accordingly, it was determined that the CoA and CoGn parameters showed low values (ICC < 0.50, Table 5).  Table 4: Evaluation and comparison of the differences between the measurements obtained after the cephalometric analysis performed by the DM method and the AI-based automated method, according to the skeletal malocclusion groups.

DISCUSSION
This study, which has the quality of guiding future studies, aimed to investigate the relationship between and reliability of DM and AI-based cephalometric analysis methods, according to skeletal malocclusion classes in the sagittal dimension.
In his study, Alqahtani 6 noted that the cloud-based websites that support the cephalometric analysis method will be a practical tool because they are fast, make storage easy, require no installation, and are easily accessible on all website platforms.
In the study conducted by Kim et al, 15  it was determined that AI-based systems still need improvements. In a study conducted by Alqahtani, 6 it was determined that the AI-based online cephalometric analysis method has high reproducibility and practical use. Therefore, the usability Çoban G, Öztürk T, Hashimli N, Yağcı A -Comparison between cephalometric measurements using digital manual and web-based artificial intelligence cephalometric tracing software 20 of this method in the context of different types of malocclusion was examined in the present study. 6 However, these systems are still in the development phase, and their use in more specific areas should be investigated. Furthermore, it has been stated in the literature that the cephalometric drawings performed manually can differ between professionals as well as between repeated drawings of the same professional. 16,17 For these reasons, the importance of AI-based automatic systems in reducing application errors is increasing. In order to contribute to these studies, it was found that the measurements used in cephalometric analysis may differ when grouping was made according to skeletal malocclusion classes in the present study. In the study of Yu et al, 18 it was stated that the precision and accuracy of AI-based automatic skeletal malocclusion classification were high. However, the analyses for determining the skeletal Class were not mentioned. In the present study, it was determined that AI-based automatic cephalometric analysis can be used for different measurements, but still requires improvement.
Based on the present results, it can be concluded that when evaluating cephalometric analysis for deep learning, more samples of different skeletal malocclusions should be evaluated.
Çoban G, Öztürk T, Hashimli N, Yağcı A -Comparison between cephalometric measurements using digital manual and web-based artificial intelligence cephalometric tracing software

21
In the present study, except for SNB, NPog, U1.SN, U1.NA, L1-APog, I/I, and LLE parameters, all other parameters presented significant differences between the DM and AI methods.
However, these parameters may also differ between skeletal malocclusion groups. The correct determination of skeletal cephalometric parameters, which is the basic standard in the treatment of skeletal malocclusions, is essential for ideal treatment. 19 When the reliability levels of the two methods were compared and evaluated, it was determined that both methods are suitable for orthodontic analysis. 5,8,13 In this study, only the CoA and CoGn parameters had low reliability between methods. This was thought to be due to the difficulty of determining the condylion (Co) point. 20 These findings indicate that, unlike the long-standing DM method, 2,5,12,19 the AI-based automatic method still requires further development.
Çoban G, Öztürk T, Hashimli N, Yağcı A -Comparison between cephalometric measurements using digital manual and web-based artificial intelligence cephalometric tracing software

CONCLUSION
Artificial intelligence-based cephalometric analysis methods can both accelerate and facilitate orthodontic treatment planning, thanks to easy archiving and automatic processes. The present study has shown that there are significant differences between the AI-based automatic method and the DM method in most cephalometric analysis measurements. Therefore, AI-based analysis needs further development and more testing in different malocclusion groups, which may change orthodontic treatment planning. The relative measurement reliability between the two techniques was found to be high, except for the CoA and CoGn measurements. There are significant differences between the measurement quantities in the two techniques, and the AI-based technique needs to be developed in these aspects. Although there are statistically significant differences in the measurements obtained between the two methods, the authors believe that there is no "clinically significant" difference between the methods to ensure a rapid preliminary assessment of orthodontic treatment planning. It is undeniable that rapidly